Cost-effectiveness of left atrial appendage closure for stroke prevention in atrial fibrillation: a systematic review appraising the methodological quality

Background The increasing global prevalence of atrial fibrillation (AF) has led to a growing demand for stroke prevention strategies, resulting in higher healthcare costs. High-quality economic evaluations of stroke prevention strategies can play a crucial role in maximising efficient allocation of resources. In this systematic review, we assessed the methodological quality of such economic evaluations. Methods We searched electronic databases of PubMed, EMBASE, CINAHL, Cochrane Central Register of Controlled Trials, and Econ Lit to identify model-based economic evaluations comparing the left atrial appendage closure procedure (LAAC) and oral anticoagulants published in English since 2000. Data on study characteristics, model-based details, and analyses were collected. The methodological quality was evaluated using the modified Economic Evaluations Bias (ECOBIAS) checklist. For each of the 22 biases listed in this checklist, studies were categorised into one of four groups: low risk, partial risk, high risk due to inadequate reporting, or high risk. To gauge the overall quality of each study, we computed a composite score by assigning + 2, 0, − 1 and − 2 to each risk category, respectively. Results In our analysis of 12 studies, majority adopted a healthcare provider or payer perspective and employed Markov Models with the number of health states varying from 6 to 16. Cost-effectiveness results varied across studies. LAAC displayed a probability exceeding 50% of being the cost-effective option in six out of nine evaluations compared to warfarin, six out of eight evaluations when compared to dabigatran, in three out of five evaluations against apixaban, and in two out of three studies compared to rivaroxaban. The methodological quality scores for individual studies ranged from 10 to − 12 out of a possible 24. Most high-risk ratings were due to inadequate reporting, which was prevalent across various biases, including those related to data identification, baseline data, treatment effects, and data incorporation. Cost measurement omission bias and inefficient comparator bias were also common. Conclusions While most studies concluded LAAC to be the cost-effective strategy for stroke prevention in AF, shortcomings in methodological quality raise concerns about reliability and validity of results. Future evaluations, free of these shortcomings, can yield stronger policy evidence. Supplementary Information The online version contains supplementary material available at 10.1186/s12962-023-00486-0.

Specify the inclusion and exclusion criteria for the review.Yes Information sources Specify the information sources (e.g.databases, registers) used to identify studies and the date when each was last searched.

Yes
Risk of bias Specify the methods used to assess risk of bias in the included studies.

Yes Synthesis of results
Specify the methods used to present and synthesise results.Yes RESULTS

Included studies
Give the total number of included studies and participants and summarise relevant characteristics of studies.

Synthesis of results
Present results for main outcomes, preferably indicating the number of included studies and participants for each.If meta-analysis was done, report the summary estimate and confidence/credible interval.If comparing groups, indicate the direction of the effect (i.e. which group is favoured).

Limitations of evidence
Provide a brief summary of the limitations of the evidence included in the review (e.g.study risk of bias, inconsistency and imprecision).

Yes
Interpretation Provide a general interpretation of the results and important implications.Yes OTHER Funding Specify the primary source of funding for the review.
No, but details are available within the main text.

Registration
Provide the register name and registration number.
No, but details are available within the main text.

METHODS
Eligibility criteria 5 Specify the inclusion and exclusion criteria for the review and how studies were grouped for the syntheses.'Data sources, search strategy and study selection for the review' sub-section Information sources 6 Specify all databases, registers, websites, organisations, reference lists and other sources searched or consulted to identify studies.Specify the date when each source was last searched or consulted.
'Data sources, search strategy and study selection for the review' sub-section Search strategy 7 Present the full search strategies for all databases, registers, and websites, including any filters and limits used.

Supplementary file
Selection process 8 Specify the methods used to decide whether a study met the inclusion criteria of the review, including how many reviewers screened each record and each report retrieved, whether they worked independently, and if applicable, details of automation tools used in the process.
'Data sources, search strategy and study selection for the review' sub-section

Data collection process
9 Specify the methods used to collect data from reports, including how many reviewers collected data from each report, whether they worked independently, any processes for obtaining or confirming data from study investigators, and if applicable, details of automation tools used in the process.
'Data extraction' sub-section Data items 10a List and define all outcomes for which data were sought.Specify whether all results that were compatible with each outcome domain in each study were sought (e.g. for all measures, time points, analyses), and if not, the methods used to decide which results to collect.13d Describe any methods used to synthesize results and provide a rationale for the choice(s).If meta-analysis was performed, describe the model(s), method(s) to identify the presence and extent of statistical heterogeneity, and software package(s) used.
13e Describe any methods used to explore possible causes of heterogeneity among study results (e.g.subgroup analysis, metaregression).
Not applicable 13f Describe any sensitivity analyses conducted to assess robustness of the synthesized results.

Not applicable
Reporting bias assessment 14 Describe any methods used to assess risk of bias due to missing results in a synthesis (arising from reporting biases).

Not applicable
Certainty assessment 15 Describe any methods used to assess certainty (or confidence) in the body of evidence for an outcome.Not applicable

Study selection 16a
Describe the results of the search and selection process, from the number of records identified in the search to the number of studies included in the review, ideally using a flow diagram.
Sub-section 'Study selection' and Figure 1 16b Cite studies that might appear to meet the inclusion criteria, but which were excluded, and explain why they were excluded.19 For all outcomes, present, for each study: (a) summary statistics for each group (where appropriate) and (b) an effect estimate and its precision (e.g.confidence/credible interval), ideally using structured tables or plots.Quality assessment of primary studies using ECOBIAS checklist How the gradings were offered

Narrow perspective bias
• Was a societal perspective adopted?
• If not, has a different perspective been justified?
Studies that either used a societal perspective or justified their chosen perspective were graded as having a low risk of bias.Conversely, studies that did neither were graded as having a high risk of bias.Among the reviewed studies, only one study 2 used a societal perspective, while six reported a healthcare provider perspective [3][4][5][6][7][8] and the remainder reported a healthcare payer perspective.Only the economic evaluation by Ontario Health 9 provided a justification for the perspective used and was therefore graded as having a partial risk for the narrow perspective bias.

Inefficient comparator bias
• Was the best alternative chosen as comparator?• Was current practice chosen as a comparator?Have all comparators been described in sufficient detail?
The best alternative or current practice for stroke prophylaxis was determined as the therapeutic modalities most widely used within the relevant jurisdiction, a practice well-established in guidelines governing economic evaluations 10,11 .Consequently, studies that compared LAAC with all commonly used oral anticoagulants (warfarin, dabigatran, apixaban, and rivaroxaban) for a base case population without contraindications for oral anticoagulants were graded as having a low risk for this bias.All the studies provided sufficient detail in describing the chosen comparator.

Cost-measurement omission bias
• Were all costs relevant to the disease and intervention identified and considered?
None of the studies included implementation costs for LAAC or oral drugs, a significant category of costs, especially for LAAC, which requires specialized staff, equipment, and space.Consequently, all studies were graded as having a high risk for this bias.
All studies except one 8 presented the items/events they costed with relevant references in a table.However, some studies were not exhaustive in listing costs.For example, four studies 5,[12][13][14] did not detail how peri-procedural complications of the LAAC procedure were costed, despite including these complications in the model structure.Additionally, some studies 5,6,8,[12][13][14][15] did not include follow-up care for LAAC, which is a significant omission.

Intermittent data collection bias
• Was the resource use measured continuously?
The checklist emphasizes the importance of having insight into all resource use (costs) throughout the entire follow-up period 16 .Previous studies suggested estimating total costs based on intermittently collected data, such as three months within a one-year period 17,18 , but it is now known to lead to bias 19 .
Studies that accounted for follow-up care costs did not report whether the data were collected continuously or intermittently, making them vulnerable to receiving a high-risk grading due to inadequate reporting.

Invalid valuation bias
• Is the price calculation presented in a detailed manner?• Have reference prices been used?Studies often included references for the costs used in the economic model, but they did not provide detailed calculations.We graded studies as having a low risk for this bias if they included valid references, such as reference prices and costs from previous literature, as recommended 1 .Two studies received a partial risk grading for this bias.One study 3 did not use local reference prices when available, and another study 8 did not provide references for all the cost items used in the model.adequately justified?risk of disease recurrence and complications with age in AF.However, one study 3 adopted a 10-year horizon, starting with a cohort aged 65 years, potentially insufficient to capture all relevant costs and outcomes.This shorter horizon lacked justification, leading to a high-risk bias rating.

Bias related to data identification
• Are the methods of data identification transparent?• Are all choices justified adequately?• Do the input parameters come from high quality and well-designed studies?
Among the reviewed studies, only one study 2 provided information on how data sources were identified, and none of the studies provided justifications for their choice of data sources.It's worth noting that the determination of data source quality can be subjective.However, concerns were raised regarding the methodological quality of the PROTECT-AF and PREVAIL trials [23][24][25] when these trials were exclusively used as sources for clinical parameters in LAAC.
The majority of studies received a high-risk rating for inadequate reporting because they could not satisfactorily address two or more of the three key questions outlined in the left column of the assessment tool.
Bias related to baseline data • Are probabilities, for example, based on natural history data?• Is transformation of rates into transition probabilities done accurately?
All studies, except for Labori and colleagues 2 extensively utilized data from the PROTECT-AF and/or PREVAIL trials, which compared LAAC with warfarin.Labori and colleagues chose not to use data from these trials due to differences in the study populations.
Similar to the approach taken by Labori et al., many other studies also encountered disparities between the characteristics of the model population and those of the trial population, such as differences in contraindications to oral anticoagulants (OAC), mean stroke risk, and bleeding risk.Despite these discrepancies, they relied on trial data for model inputs.Additionally, none of the studies provided adequate descriptions of how the rates from the trials were transformed into transition probabilities.This lack of detail made them susceptible to receiving a high-risk rating due to inadequate reporting.
Bias related to treatment effects  9 which conducted a meta-analysis utilizing recent data extracted from trials, observational studies, and registries 26 .Notably, some studies 5,8,15 opted to use only PROTECT-AF trial data to derive clinical inputs for LAAC, excluding PREVAIL trial data.This choice may have introduced bias, as the PREVAIL trial failed to demonstrate the non-inferiority of LAAC against warfarin for ischaemic stroke prevention 27 .Unfortunately, none of the studies provided detailed justifications for their extrapolations or explored alternative assumptions regarding extrapolations.Consequently, all studies received a high-risk rating due to inadequate reporting.
Bias related to quality-of-life weights (utilities) • Are the utilities incorporated appropriate for the specific decision problem?
Some studies provided only the mean utility score used for model input parameters 3,4 , while others offered brief descriptions of the methods used to estimate this mean score 5,[12][13][14] .A few studies mentioned using quality-of-life short form-12 data collected during the Watchman ASAP trial 5 and PROTECT-AF trial 13,14 to derive QALY estimates for LAAC.In contrast, three studies [6][7][8] reported a lack of quality-of-life estimates for LAAC and instead utilized relevant data from percutaneous coronary interventions.However, despite differences in data sources, all studies received a low-risk rating for this bias, as the utilities incorporated were deemed appropriate for the specific decision problem.

No transparent data incorporation bias
• Is the process of data incorporation transparent?• Are all data and their sources described in detail?
All studies comparing LAAC with a novel OAC, except for the study by Labori and colleagues which employed a meta-analysis, utilized indirect comparison techniques to derive clinical inputs for LAAC.This was because the available pivotal trials had compared LAAC only with warfarin.However, only one study 9 provided a clear presentation of how this indirect comparison was conducted.Moreover, although three out of ten studies 5,7,8 reported using patient-level Markov microsimulation models, none of them specified transition probabilities for the events listed in their tables based on patient-level data.As a result, these studies were assigned a high-risk bias rating due to inadequate reporting.Limited scope bias • Have the four principles of uncertainty (methodological, structural, heterogeneity, parameter) been considered?
All included studies presented and discussed probabilistic sensitivity analyses, which are typically employed to evaluate parameter uncertainty.However, they did not address methodological or structural uncertainty in their analyses.This omission exposed the studies to a high risk of limited scope bias due to inadequate reporting.
Bias related to internal consistency.
• Has internal consistency in terms of mathematical logic been evaluated?
None of the primary studies reported an analysis for the internal consistency of the model, as recommended by best practice guidelines 1 .

Table s1 .
Prisma for Abstracts Checklist

Table 1 Results
Report which of the following are publicly available and where they can be found: template data collection forms; data extracted from included studies; data used for all analyses; analytic code; any other materials used in the review.A meta-analysis was not performed.
of syntheses 20a For each synthesis, briefly summarise the characteristics and risk of bias among contributing studies.Sub-sections 'Main study and economic model characteristics' and 'methodological quality assessment'20b Present results of all statistical syntheses conducted.If meta-analysis was done, present for each the summary estimate and its precision (e.g.confidence/credible interval) and measures of statistical heterogeneity.If comparing groups, describe the direction of the effect.From: Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al.The PRISMA 2020 statement: an updated guideline for reporting systematic reviews.BMJ 2021;372:n71.doi:10.1136/bmj.n71Formore information, visit: http://www.prisma-statement.org/

Table s3 .
Search strategy of the review

Table s4 .
Quality assessment of primary studies using ECOBIAS checklist Studies that compared LAAC with novel oral anticoagulants (OAC) employed various indirect comparison techniques to derive treatment effects.Most studies converted treatment effects for LAAC and warfarin from pivotal trials, with the exception of one study